Informed Exploration: A Satisficing Approach to Q-learning

In the design of robots and automated systems, it is often desirable to endow decision-making agents with an ability to perform self-governed learning. Since many environments produce situations that have not been anticipated, even by the most clever designers, an agent must be endowed with a flexibility to explore options and decide issues that extend beyond rote execution of designer commands. With this research vision as the backdrop, we present a modification to traditional Q-learning that allows a designer to optimally restrict the set of feasible exploration strategies while simultaneously permitting an agent to freely explore within these restrictions. This approach uses a rule of satisficing decision-making that balances the tradeoff between exploiting current knowledge and exploring possible alternatives.